Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update GSI analysis jobs to use COMIN/COMOUT #3092

Merged
merged 27 commits into from
Dec 12, 2024
Merged

Update GSI analysis jobs to use COMIN/COMOUT #3092

merged 27 commits into from
Dec 12, 2024
+206 −197

Conversation

mingshichen-noaa
Copy link
Contributor

@mingshichen-noaa mingshichen-noaa commented Nov 12, 2024 •
edited
Loading

Description

NCO has requested that each COM variable specify whether it is an input or an output. This completes that process for the global jgdas enkf select obs job and relevant .JGLOBAL_ATMOS_ANALYSIS and JGLOBAL_ATMOS_ANALYSIS_CALC jobs.

Refs #2451

Type of change

  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO

How has this been tested?

  • Clone and build on RDHPCS
  • Cycled tests on Hercules
  • Forecast-only tests on Hercules

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • I have made corresponding changes to the documentation if necessary

github-advanced-security[bot]
github-advanced-security bot found potential problems Nov 12, 2024
View reviewed changes
jobs/JGDAS_ENKF_SELECT_OBS Fixed Show resolved Hide resolved
@mingshichen-noaa mingshichen-noaa self-assigned this Nov 12, 2024
@mingshichen-noaa mingshichen-noaa added the CI/CD Issue related to CI/CD label Nov 12, 2024
aerorahul
aerorahul reviewed Nov 12, 2024
View reviewed changes
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good. I am no longer familiar w/ the IN and OUT of EnKF related data, so would appreciate a review from @CatherineThomas-NOAA @RussTreadon-NOAA

KateFriedman-NOAA
KateFriedman-NOAA reviewed Nov 13, 2024
View reviewed changes
Copy link
Member

@KateFriedman-NOAA KateFriedman-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Please resolve the shellcheck warning in jobs/JGDAS_ENKF_SELECT_OBS and then I can approve.

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA removed the CI/CD Issue related to CI/CD label Nov 13, 2024
aerorahul
aerorahul previously approved these changes Nov 13, 2024
View reviewed changes
Copy link
Contributor

@aerorahul aerorahul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me.

KateFriedman-NOAA
KateFriedman-NOAA previously approved these changes Nov 14, 2024
View reviewed changes
Copy link
Member

@KateFriedman-NOAA KateFriedman-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @mingshichen-noaa !

@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Nov 15, 2024
@emcbot emcbot removed the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Nov 15, 2024
aerorahul
aerorahul reviewed Dec 9, 2024
View reviewed changes
jobs/JGLOBAL_ATMOS_ANALYSIS_CALC Outdated

YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_ATMOS_ANALYSIS COM_ATMOS_RESTART
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mingshichen-noaa
This line is causing the failure. Please fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul

After I checked codes in jobs/JGLOBAL_ATMOS_ANALYSIS_CALC from mingshichen-noaa:feature/jgdas_enkf_select_obs_com_in_out, the codes had been modified as follows:
30
31 RUN=${rCDUMP} YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
32 COMIN_OBS:COM_OBS_TMPL
33
34 YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
35 COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL
36 COMIN_ATMOS_RESTART:COM_ATMOS_RESTART_TMPL
37
38 RUN=${GDUMP} YMD=${gPDY} HH=${gcyc} declare_from_tmpl -rx
39 COMIN_OBS_PREV:COM_OBS_TMPL
40 COMIN_ATMOS_HISTORY_PREV:COM_ATMOS_HISTORY_TMPL
41

@mingshichen-noaa
Copy link
Contributor Author

@aerorahul
Do you think whether I should make a change on line 39 in jobs/JGDAS_ENKF_SELECT_OBS
from
39 declare -rx COMOUT_ATMOS_ANALYSIS_ENS="${COMOUT_ATMOS_ANALYSIS}"
into
39 declare -rx COMOUT_ATMOS_ANALYSIS_ENS:COM_ATMOS_ANALYSIS_TMPL

aerorahul
aerorahul reviewed Dec 10, 2024
View reviewed changes
jobs/JGLOBAL_ATMOS_ANALYSIS_CALC Outdated

YMD=${PDY} HH=${cyc} declare_from_tmpl -rx COM_ATMOS_ANALYSIS COM_ATMOS_RESTART
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
YMD=${PDY} HH=${cyc} declare_from_tmpl -rx \

This is the fix needed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul
I have changed line 34 as follows:
34 YMD=${PDY} HH=${cyc} declare_from_tmpl -rx
35 COMOUT_ATMOS_ANALYSIS:COM_ATMOS_ANALYSIS_TMPL
36 COMIN_ATMOS_RESTART:COM_ATMOS_RESTART_TMPL

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aerorahul
My changes are located the branch: mingshichen-noaa:feature/jgdas_enkf_select_obs_com_in_out

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please push them

@WalterKolczynski-NOAA
Copy link
Contributor

@mingshichen-noaa Please rewind the dead jobs in your experiment so it can continue with the fix.

@WalterKolczynski-NOAA WalterKolczynski-NOAA changed the title Update JGDAS ENKF SELECT OBS JGLOBAL_ATMOS_ANALYSIS and JGLOBAL_ATMOS_ANALYSIS_CALC jobs Update GSI analysis jobs to use COMIN/COMOUT Dec 10, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA added the CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera label Dec 11, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Dec 11, 2024
@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA added CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Dec 11, 2024
@emcbot emcbot added CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera and removed CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera labels Dec 11, 2024
@mingshichen-noaa
Copy link
Contributor Author

@aerorahul & @WalterKolczynski-NOAA ,
I have checked the testing states of C96C48_hybatmDA are SUCCEEDED. Please have a look at the testing results at
/work2/noaa/global/mchen/test/jobs/JGDAS_ENKF_SELECT_OBS/EXPDIR/C96C48_hybatmDA

@emcbot emcbot added CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Dec 11, 2024
@emcbot
Copy link

emcbot commented Dec 11, 2024

CI Passed on Hera in Build# 2
Built and ran in directory /scratch1/NCEPDEV/global/CI/3092


Experiment C48mx500_3DVarAOWCDA_df83c7bf Completed 2 Cycles: *SUCCESS* at Wed Dec 11 20:12:45 UTC 2024
Experiment C48_ATM_df83c7bf Completed 2 Cycles: *SUCCESS* at Wed Dec 11 20:12:45 UTC 2024
Experiment C48mx500_hybAOWCDA_df83c7bf Completed 2 Cycles: *SUCCESS* at Wed Dec 11 20:25:00 UTC 2024
Experiment C96_S2SWA_gefs_replay_ics_df83c7bf Completed 1 Cycles: *SUCCESS* at Wed Dec 11 20:49:59 UTC 2024
Experiment C96C48_hybatmaerosnowDA_df83c7bf Completed 3 Cycles: *SUCCESS* at Wed Dec 11 21:38:16 UTC 2024
Experiment C96_atm3DVar_df83c7bf Completed 3 Cycles: *SUCCESS* at Wed Dec 11 21:44:04 UTC 2024
Experiment C96C48_hybatmDA_df83c7bf Completed 3 Cycles: *SUCCESS* at Wed Dec 11 21:44:18 UTC 2024
Experiment C96C48_ufs_hybatmDA_df83c7bf Completed 3 Cycles: *SUCCESS* at Wed Dec 11 22:26:50 UTC 2024
Experiment C48_S2SW_df83c7bf Completed 2 Cycles: *SUCCESS* at Wed Dec 11 22:27:45 UTC 2024
Experiment C48_S2SWA_gefs_df83c7bf Completed 1 Cycles: *SUCCESS* at Wed Dec 11 23:05:51 UTC 2024

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress and removed CI-Wcoss2-Building **Bot use only** CI testing is cloning/building on WCOSS labels Dec 12, 2024
@WalterKolczynski-NOAA
Copy link
Contributor

CI Tests set up to run in /lfs/h2/emc/ptmp/walter.kolczynski/PR/PR_3092/RUNTESTS on WCOSS

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully and removed CI-Hercules-Ready **CM use only** PR is ready for CI testing on Hercules CI-Wcoss2-Running **Bot use only** CI testing on WCOSS for this PR is in-progress labels Dec 12, 2024
@WalterKolczynski-NOAA WalterKolczynski-NOAA merged commit 7090cff into NOAA-EMC:develop Dec 12, 2024
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DavidHuber-NOAA DavidHuber-NOAA DavidHuber-NOAA left review comments

@aerorahul aerorahul aerorahul left review comments

@WalterKolczynski-NOAA WalterKolczynski-NOAA WalterKolczynski-NOAA approved these changes

@KateFriedman-NOAA KateFriedman-NOAA KateFriedman-NOAA left review comments

Assignees

@mingshichen-noaa mingshichen-noaa

Labels
CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Wcoss2-Passed **Bot use only** CI testing on WCOSS for this PR has completed successfully
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@mingshichen-noaa @emcbot @aerorahul @KateFriedman-NOAA @WalterKolczynski-NOAA @DavidHuber-NOAA @TerrenceMcGuinness-NOAA